Skip to main content


         This documentation site is for previous versions. Visit our new documentation site for current releases.      
 

Creating and running an Extract rule

Updated on April 6, 2022

After you enable the built-in Business Intelligence Exchange (BIX) ruleset on Pega Platform, you can create and run Extract rules to extract specific class data from your application.

Create and open an Extract rule to run a manual BIX extraction process. To limit the properties included in your extract, use filter criteria to extract specific properties from a class. Filters can also exclude specific properties from an extraction and extract properties that have changed since the last extraction was run. Implementation of filter conditions can significantly increase BIX extraction performance. See step 7 and Creating and running an Extract: Filter Criteria to implement filters. For more information about BIX performance, see Optimizing BIX data extraction performance.
Note: Manual BIX extractions are not designed to be run on a production environment. To run extractions as a background process on a production environment after creating an Extract rule, see Using the Job Scheduler rule to extract data with BIX.
  1. In the header of Dev Studio, click Create SysAdminExtract.

    Alternatively, in the navigation pane of Dev Studio, click App. Right-click the name of the application from which you want to extract class data, and then select Create SysAdminExtract.

  2. In the Label field, enter a name for the Extract rule.
  3. In the Context field, select the application, class, and ruleset from which to extract data.
  4. Click Create and open.
  5. On the Definition tab, specify the output format, and then select properties to extract:
    • XML: Extract properties to an XML file according to top-level pages, page lists, and page groups with embedded XML tags.
      • For on-premises environments: The application extracts the XML file to an output directory defined in the File Specification tab in step 8.
      • For Pega Cloud environments: The application extracts the XML file to the Pega Cloud File Storage Repository.
    • Comma Separated Value (CSV): Extract properties to comma-separated values in separate text files according to top-level pages, pages lists, page groups, and value properties.
      • For on-premises environments: The application extracts CSV files to an output directory defined in the File Specification tab in step 8.
      • For Pega Cloud environments: The application extracts CSV files to the Pega Cloud File Storage Repository. The CSV format for Pega Cloud environments also adds a password protection option below the format drop-down.
    • Database Schema: Extract properties directly into an existing relational database outside of the application. The extraction process forms top-level pages, page lists, page groups, and value properties from a selected class into separate tables on the database. Identify the external database instance on the File Specification tab in step 8.

      Important: You must have insert and view-metadata privileges for the database into which you want to insert the Database Schema. Contact your Database Administrator for details. Pega Platform does not support extracting to a Teradata database due to performance issues with the metadata queries that are common in BIX extractions.
      For more information about creating an external database instance, see Creating and updating external database instances with JDBC URLs.
    1. In Property, at the class level, click the properties icon.
    2. In the Properties Configuration Window, in the From Class, select the check box of the page lists, page groups, and single value properties to extract, then click Submit.
    3. After selecting a page list or page group to extract, click the arrow icon next to the class to display the selected page lists or page groups.
    4. Click the properties icon next to that page list or page group to further specify nested pages or single value properties to extract.
    5. For single value properties, format how the extracted output appears with the following options varying on output type:
      • For CSV extractions: Map single-value properties to a name in the CSV file. Do not duplicate names.
      • For Database Schema extractions: In To Table, provide a name for the extracted properties table in the external database for the extraction process. Do not use whitespace or duplicate names.
    6. In the Property Configurations form, click Submit. Return to step 5b as necessary. XML formats can extract all properties from a selected class with option Get all properties below the format drop-down.
  6. Click Save.
  7. Optional: On the Filter Criteria tab, select filter conditions using properties of the primary class. For more information, see Creating and running an Extract: Filter Criteria.
  8. In the File Specification tab, specify the directory details for the specific output file format you want to use for the extraction process:

    On-premises extraction directories and file naming options

    Output formatFields
    XML
    • XML Schema Directory - Specify a local directory in which to place the XML schema file.
    • XML Schema File Name - Enter a name for the XML schema files.
    • XML File Output Directory - Specify the target directory to place the XML output files.
    • XML Output File Name - Specify the target directory in which to place the XML output files. Pega Platform automatically adds .XML and.XSD extensions to the file name.

    Pega Platform supports appending the following wild-card formatting strings to the output file name:

    • %i - (RunIdentifier) Adds a unique run-identifier to the extract file.
      • Pega Platform generates a run-identifier consisting of the pzInsKey of the Extract rule followed by the sequence number which increments after each extraction process.
    • %s - Adds an automatically generated sequence number to each new extract. For example, BIXExtract1, BIXExtract2
    • %c - (ClassName) Adds the class to which Pega Platform applied the Extract rule (see step 3). For example, BIXExtract_PegaSample-Work
    • %r - Adds the name of the Extract rule. For example, BIXExtract_WeeklyExtract1
    • %d - (ExtractRunDate) Adds the date of the extract run.
    • %t - (ExtractRunDateTime) Adds the date and time of the extract run.

    For example, specifying the output file name BIXExtract_%s_%c_%r generates in the output file name as PegaExtract_1_PegaSample-Work_WeeklyExtract.

    Note: Use %d and %t when running multiple extracts to get unique filenames and avoid Pega Platform overwriting previous extract files with the same name in the specified directory
    CSVSpecify the target directory for the CSV extract files.
    Database SchemaSpecify a JDBC URL to an existing external database instance into which to insert the database schema.

    You must have the insert and view-metadata privileges for the database into which you want to insert the database schema.

    Pega Cloud extraction directories

    Output formatFields
    XMLXML & CSV extractions place the extract files into your Pega Cloud File Storage environment repository.

    To use a subfolder in your BIX folder to place your XML & CSV extract files, enter its name in the Subfolder field.

    CSV
    Database SchemaSpecify a JDBC URL that links to an existing external target database instance into which to insert the database schema.

    You must have the insert and view-metadata privileges for the database into which you want to insert the database schema.

    On-premises manifest extraction directory and file naming options

    Output formatFields
    XML
    • Manifest XML file output directory - Specify a local target directory to receive the XML manifest file.
    • Manifest XML output file name - Enter a name for the XML manifest files. Pega Platform automatically adds .xml and .xsd extensions to the file name.

    Pega Platform supports appending the following wild-card formatting strings to the output filename:

    • %i - (RunIdentifier) Adds a unique run-identifier to the extract file.
      • Pega Platform generates a run-identifier consisting of the pzInsKey of the Extract rule followed by the sequence number which increments after each extraction process.
    • %s - Adds automatically generated sequence number of each new extract. For example, BIXExtract1, BIXExtract2
    • %c - (ClassName) Adds the class to which Pega Platform applied the Extract rule (see step 3). For example, BIXExtract_PegaSample-Work
    • %r - Adds the name of the Extract rule. For example, BIXExtract_WeeklyExtract1
    • %d - (ExtractRunDate) Adds the date of the extract run.
    • %t - (ExtractRunDateTime) Adds the date and time of the extract run.

    For example, specifying the output file name BIXExtract_%s_%c_%r generates in the output file name as PegaExtract_1_PegaSample-Work_WeeklyExtract.

    Note: Use %d and %t when running multiple extracts to get unique filenames and avoid Pega Platform overwriting previous extract files with the same name in the specified directory.
    CSVSpecify the target directory for the CSV manifest file.
    Database SchemaSpecify a JDBC URL to an existing external database instance into which to insert the database schema.

    You must have the insert and view-metadata privileges for the database into which you want to insert the database schema manifest files.

    Pega Cloud manifest extraction directories

    Output formatFields
    XMLXML and CSV extractions place the manifest files in the Pega Cloud File Storage repository.
    CSV
    Database SchemaSpecify a JDBC URL to an existing external database instance into which to insert the database schema manifest files.

    You must have insert and view-metadata privileges for the database into which you want to insert the database schema manifest files.

    1. Click Test Connectivity to confirm local application server filepath validity for XML and CSV extractions (on-premises only) and database connectivity for database schema extractions.
    2. Optional: In Include check total property, enter a property to use as a checksum SQL function for the extraction process. This option is not available for high-throughput CSV extractions. For more information, see Optimizing BIX data extraction performance.
    3. Click Save to apply the directories and external database location that you select.
  9. In the Parameters tab, create parameters that populate the clipboard at runtime. You use these parameters to customize BIX extractions by specifying them in the Filter Criteria tab.
    1. Enter the Name and Description for the parameter.
    2. In Data Type, enter the value type of the parameter
    3. In Default value, enter the parameters default value.
    4. Click Save to apply parameters.
    5. To specify these parameters in a Filter Criteria, enter these parameters in the Filter Criteria tab. For details, see Creating and running an Extract: Filter Criteria.
  10. Optional: In the Pages & Classes tab, create pages that populate the clipboard at runtime. You use these Pages & Classes to customize data extractions by specifying them in the Filter Criteria tab.
    1. In Page name, identify a clipboard page with the format pyWorkPage.pyUserPage.
    2. In the Class field, select the class of the specified page.
    3. Click Save to apply the page and class.
    4. To specify these pages and classes in a Filter Criteria, enter these pages and classes in the Filter Criteria tab. For details, see Creating and running an Extract: Filter Criteria.
  11. In the Execution History tab, view a list of concurrently running and previously run BIX extractions by clicking on each row.
  12. To manually run a BIX extraction process, in the rule form toolbar, click ActionsRun.
  13. In the Label to use for this run field, enter a unique identifier for this BIX extraction. Extract files contain the unique identifier to reference the Extract rule that ran the process. For details, see Unique run identifier.
  14. For XML and CSV extracts, click Perform Extract.
  15. For Database Schema extractions, perform the following tasks to generate a DDL for new extracts, changes to an existing extraction, or after a BIX upgrade:
    1. Click Generate schema DDL to generate a DDL file to create a formatted database table for the extraction process.
    2. If you chose to generate a manifest and store in a database schema, click Generate manifest schema DDL for the manifest table.
    3. Click Perform Extract.
      • For Pega Cloud deployments, a URL appears for the DDL file.
      • For on-premises deployments, a directory field appears for the system to download a DDL file to a location on the application server.
      • Share the DDL file with your database administrator so that the administrator can format the external database table for extractions.
  16. For Database Schema extractions, perform the following tasks to run the extraction after completing step 15:
    1. To run manual extractions to a database schema, repeat step 15 with the Generating schema DDL and the Generate manifest schema DDL check box cleared. BIX extractions through the Job Scheduler for database schema formats are now available to run after implementing schema DDL for a target database.
    2. In the Number of records to update at a time field, enter the number of records to insert to the target database in a batch.
    3. Click Perform Extract.
  17. Repeat step 12 to 16 as applicable to run additional manual extraction processes.
    • To run a BIX extraction process with additional filter criteria (such as when defining a filter for a specific table that is updated from the last extraction), repeat steps 7a to 7d, then return to step 12.
What to do next: Retrieve your BIX extract files from their default save location:
  • For Pega Cloud Services clients using XML or CSV formats, your BIX extract files are saved in the Pega Cloud Services File Storage repository. For more information, see Procedures for BIX extractions in Pega Cloud environments.
  • For on-premises clients using XML or CSV formats, your BIX extract files are saved in the local directory that you specify.
  • For database schema extractions, refer to your external database instance to view the extraction table data.

Related articles

Overview of BIX extractions in Pega Cloud environmentsOverview of BIX extractions in on-premises environments

Tags

Pega Platform 8.6 Data Integration

Have a question? Get answers now.

Visit the Support Center to ask questions, engage in discussions, share ideas, and help others.

Visit the Support Center

Did you find this content helpful?

Yes
No

Want to help us improve this content?
Suggest an edit

We'd prefer it if you saw us at our best.

Pega.com is not optimized for Internet Explorer. For the optimal experience, please use:

Close Deprecation Notice
Contact us